Image display device using combined Retinex processing

ABSTRACT

An image whose visibility has been more suitably improved is obtained. The device includes an image input unit which inputs an image, an image processing unit which performs Retinex processing with respect to the input image input by the image input unit and performs image signal generation to generate a new image signal based on information concerning a color of the input image and information of an absolute value or Y-value of a color space vector of the image signal having undergone the Retinex processing, and a display unit which displays an image based on the image signal having undergone image processing by the image processing unit.

TECHNICAL FIELD

The present invention relates to an image processing technique.

BACKGROUND ART

Patent literature 1 discloses a background art in this technical field.According to this literature, in Multi Scale Retinex Processing, acomposite blur image is generated by selecting, for each pixel, any of aplurality of blur images with different degrees of blurring, which aregenerated by a plurality of peripheral functions with different scales,in accordance with the pixel value levels of an original image to beprocessed. The literature describes that a low-pass filter is applied tothe composite blur image to prevent the unnatural discontinuity ofboundaries and perform Retinex Processing (see the abstract).

CITATION LIST Patent Literature

PTL 1: Japanese Patent Application Laid-Open No. 2005-004506

SUMMARY OF INVENTION Technical Problem

An image signal has various parameters such as luminances, colors, andfrequency components as parameters representing the properties of a shotobject. The values of these parameters differ as to different scenes ofthe image. In order to display an image with good visibility, it isnecessary to perform image correction by changing properties such ascontrast correction of the image in accordance with the features of theimage.

However, a technique of implementing higher performance in terms ofdynamic range compression by adjusting a plurality of scales in MSR asin patent literature 1 described above gives consideration to thecontribution of a plurality of scales to an image but gives noconsideration to the characteristics of an object. As a consequence, animage is uniformly corrected regardless of the characteristics of anobject in the image.

In addition, the technique of implementing higher performance in termsof dynamic range compression by adjusting a plurality of scales in MSRas in patent literature 1 described above gives consideration to thecontribution of a plurality of scales to an image but gives noconsideration to the contribution of differences in reflection propertyto an image.

Solution to Problems

In order to solve the above problems, an aspect of an embodiment of thepresent invention may be configured to include an image input unit whichinputs an image, an image processing unit which performs Retinexprocessing with respect to the input image input by the image input unitand performs image signal generation to generate a new image signalbased on information concerning a color of the input image andinformation of an absolute value or Y-value of a color space vector ofthe image signal having undergone the Retinex processing, and a displayunit which displays an image based on the image signal having undergoneimage processing by the image processing unit.

Advantageous Effects of Invention

The present invention can obtain an image with more suitably improvedvisibility.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of the arrangement of animage display device according to Example 1 of the present invention;

FIG. 2 is a block diagram showing an example of the arrangement of animage correction unit;

FIG. 3 is a block diagram showing an example of the arrangement of animage combining unit;

FIG. 4A is a graph showing an example of a characteristic of a firstRetinex processing unit;

FIG. 4B is a graph showing an example of a characteristic of a secondRetinex processing unit;

FIG. 4C is a graph showing an example of a characteristic of an imagecombining control signal;

FIG. 5A is a graph showing an example of the luminance histogram of animage;

FIG. 5B is a graph showing an example of the input/output characteristicof an image;

FIG. 5C is a graph showing an example of the luminance histogram of animage;

FIG. 5D is a graph showing an example of the input/output characteristicof an image;

FIG. 5E is a graph showing an example of the luminance histogram of animage;

FIG. 5F is a graph showing an example of the input/output characteristicof an image;

FIG. 6 is a graph showing an operation characteristic of a featureanalyzing unit;

FIG. 7 is a block diagram showing an example of the arrangement of aRetinex processing unit according to Example 3 of the present invention;

FIG. 8 is a block diagram showing an example of the arrangement of areflected light detection unit;

FIG. 9A is a block diagram showing an example of the arrangement of areflected light control unit;

FIG. 9B is a block diagram showing an example of the arrangement of thereflected light control unit;

FIG. 10 is a view for explaining properties of reflected light based onthe Phong reflection model;

FIG. 11A is a graph for explaining a Gaussian distribution;

FIG. 11B is a graph for explaining a luminance distribution in the formof a cosine distribution;

FIG. 11C a graph for explaining a luminance distribution in the form ofa raised cosine distribution;

FIG. 12A is a graph for explaining a specular correction gain based onthe luminance value of an image;

FIG. 12B is a graph for explaining a diffuse correction gain based onthe luminance value of an image;

FIG. 13 is a block diagram showing an example of the arrangement of animage display device according to Example 4 of the present invention;

FIG. 14 is a block diagram showing an example of the arrangement of animage correction unit;

FIG. 15 is a block diagram showing an example of the arrangement of animage combining unit;

FIG. 16 is a graph showing an example of a control characteristic basedon illuminance;

FIG. 17 is a block diagram showing an example of the arrangement of animage display device according to Example 5 of the present invention;

FIG. 18 is a view showing an example of a setting menu screen accordingto Example 5 of the present invention;

FIG. 19 is a block diagram showing an example of the arrangement of animage display device according to Example 6 of the present invention;

FIG. 20 is a block diagram showing an example of the arrangement of animage correction unit;

FIG. 21 is a block diagram showing an example of the arrangement of anoutput image generation unit;

FIG. 22 is a view showing an example of an image signal level;

FIG. 23 is a block diagram showing an example of the arrangement of animage display device according to Example 7 of the present invention;

FIG. 24 is a block diagram showing an example of the arrangement of animage correction unit;

FIG. 25 is a block diagram showing an example of the arrangement of anoutput image generation unit;

FIG. 26 is a block diagram showing an example of the arrangement of animage correction unit according to Example 6 of the present invention;and

FIG. 27 is a block diagram showing an example of the arrangement of animage correction unit according to Example 8 of the present invention.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will be described below withreference to the accompanying drawings. The present invention is not,however, limited to these embodiments. Note that the same referencenumerals denote the same members in the respective drawings forexplaining the embodiments, and a repetitive description of them will beomitted.

Example 1

Example 1 will exemplify an image display device, as the arrangement ofa projector, which decomposes an image for each reflection property oflight and performs image correction. Although a front projector will beexemplified below, its form may include a rear projection television. Inaddition, Example 1 may include a display device using a direct-viewflat-panel display such as a liquid crystal display, plasma display ororganic EL display, which does not perform enlarged projection on thepanel. This applies to any of Examples described below.

FIG. 1 is a block diagram showing an example of the arrangement of animage display device according to Example 1.

This image display device includes an input signal processing unit 11which receives an image input signal 10 and converts it into an internalimage signal 12 by using, for example, a compressed image signaldecoder, IP conversion, or scaler, an image correction unit 100 whichreceives the internal image signal 12, a timing control unit 14 whichreceives a corrected image signal 13 and generates a display controlsignal 15 based on the corrected image signal and horizontal/verticalsynchronization signals for a display screen, and an optical systemdevice 200 which displays an image.

The optical system device 200 includes a light source 203 which appliesa light beam to project an image onto the screen, a panel 202 whichgenerates a projection image by receiving the display control signal 15and adjusting the tone of the light beam from the light source 203 foreach pixel, and a lens 201 for enlarged projection of a projection imageon the screen.

Note that when the image display device is a direct-view flat-paneldisplay such as a liquid crystal display, plasma display, or organic ELdisplay, the lens 201 of the optical system device 200 is not required.The user will directly view the panel 202.

FIG. 2 shows an example of the arrangement of the image correction unit100. A first Retinex processing unit 20 and a second Retinex processingunit 22 perform image processing based on a Retinex theory with respectto the internal image signal 12 and respectively output a firstcorrected image signal 21 and a second corrected image signal 23.

In this case, the Retinex theory explains human visual characteristicssuch as color constancy and brightness constancy. According to thistheory, it is possible to separate illumination light components from animage and extract reflected light components from the image.

When performing image correction processing based on the Retinex theory,therefore, it is possible to obtain an image with high visibility evenfrom an image in a dark room or against bright light by removingillumination light components which degrade the visibility of an objectsuch as a human figure in the image and extracting reflected lightcomponents. This makes it possible to suitably compress even a dynamicrange that looks natural to the human eye by using digital tones.

The Retinex theory includes many models depending on techniques ofestimating illumination light components and reflected light components.For example, reference literature 1 described below compares McCann99,PSEUDO, Poisson, and QP models.

In addition, Retinex that extracts reflected light components whileestimating that local illumination light components complying with aGaussian distribution will be called Center/Surround (to be referred toas C/S hereinafter) Retinex. Models typified by this Retinex include,for example, Single Scale Retinex model (to be referred to as SSRhereinafter) and Multiscale Retinex model (to be referred to as MSRhereinafter).

SSR is a model (see, for example, reference literature 2 describedbelow) for extracting luminance components of reflected lightcorresponding to one scale from an image. MSR is a model (see, forexample, reference literature 3 described below) obtained by extendingSSR. This model is designed to extract luminance components of reflectedlight corresponding to a plurality of scales from an image.

-   [Reference Literature 1] Yoshihiro Nozato et al., “Comparison of    Retinex Models for Hardware Implementation”, IEICE Technical Report    SIS2005-16 (2005)-   [Reference Literature 2] D. J. Jobson and G. A. Woodell, “Properties    of a Center/Surround Retinex: Part 2. Surround Design, NASA    Technical Memorandum, 110188, 1995-   [Reference Literature 3] Zia-ur Rahman, Daniel J. Jobson, and    Glenn A. Woodell, “Multiscal Retinex for Color Image Enhancement”,    ICIP'96

Assume, for example, that in Example 1, the first Retinex processingunit 20 uses a McCann99 model excelling in illumination light estimationperformance, and the second Retinex processing unit 22 uses an MSR modelexcelling in contrast correction performance. A feature analyzing unit24 analyzes the features of the internal image signal 12 and outputs afirst image combining control signal 29 and a second image combiningcontrol signal 25 to an image combining unit 26. The image combiningunit 26 outputs the corrected image signal 13 by combining the correctedimage signal 21 and the corrected image signal 23 based on the firstimage combining control signal 29 and the second image combining controlsignal 25.

FIG. 3 shows an example of the arrangement of the image combining unit26. A gain control unit 27 multiplies the corrected image signal 21 byα. A gain control unit 28 multiplies the corrected image signal 23 by(1−α). An addition unit 30 adds the resultant signals. A gain controlunit 31 multiplies the signal by β to obtain the corrected image signal13.

An example of the operation of the arrangement shown in FIGS. 1 to 3will be described next with reference to FIGS. 4A to 4C and 5A to 5F.Control using the first image combining control signal 29 in Example 1will be described first.

Referring to each of FIGS. 4A and 4B, the abscissa represents theluminance level, and the ordinate represents the gain. FIGS. 4A and 4Brespectively show examples of the gain characteristics of the first andsecond Retinex processing units 20 and 22 with respect to the luminancelevels. FIGS. 4A and 4B exemplify a case in which an McCann99 model isused for the first Retinex processing unit 20, and an MSR model is usedfor the second Retinex processing unit 22. In the example shown in FIG.4A, the first Retinex processing unit 20 using the McCann99 model has again peak g1 between luminance levels LV1 and LV2. In the example shownin FIG. 4B, the second Retinex processing unit 22 using the MSR modelhas a gain peak g2 between luminance levels LV2 and LV3.

FIG. 4C is a graph showing an example of a combining control value αbased on the first image combining control signal 29 output from thefeature analyzing unit 24 shown in FIG. 2 when the first Retinexprocessing unit 20 and the second Retinex processing unit 22 have thecharacteristics shown in FIGS. 4A and 4A. The combining control value iscontrolled in the manner shown in FIG. 4C. At a luminance level at whichthe gain of the first Retinex processing unit 20 is higher than that ofthe second Retinex processing unit 22, the combining control value α isdecreased. In contrast to this, at a luminance level at which the gainof the first Retinex processing unit 20 is lower than that of the secondRetinex processing unit 22, the combining control value α is increased.With this control, the input/output characteristic of a composite outputimage based on the first Retinex processing unit 20 and the secondRetinex processing unit 22, which is output from the addition unit 30,becomes a linear characteristic.

With the above processing, it is possible to obtain a composite imagehaving advantages of both Retinex processing using a McCann99 modelexcelling in illumination light estimation performance and Retinexprocessing using an MSR model excelling in contrast correctionperformance.

Control based on the second image combining control signal 25 in Example1 will be described next.

FIGS. 5A and 5B show an example of control based on the second imagecombining control signal 25 output from the feature analyzing unit 24.

First of all, referring to FIG. 5A, the abscissa represents theluminance level of an image, and the ordinate represents the number ofpixels in one frame. FIG. 5A is a graph expressing a distribution ateach luminance level as a histogram. In the case shown in FIG. 5A, ahistogram h1 indicates that the distribution in the range from aluminance level LV1 to a luminance level LV3 is larger than thedistributions at luminance levels equal to or lower than LV1 and equalto or higher than LV3. Note that if the distribution in the range fromthe luminance level LV1 to the luminance level LV3 is flat, a histogramh0 indicated by the alternate long and short dash line is obtained.

Referring to FIG. 5B, the abscissa represents the luminance level of aninput image, and the ordinate represents the luminance level of anoutput image. FIG. 5B shows an example of the second image combiningcontrol signal 25 output from the feature analyzing unit 24 when theluminance distribution in FIG. 5A described above is represented by thehistogram h1. This example corresponds to an input/output levelcharacteristic controlled by a gain control value β. When the luminancedistribution in FIG. 5A is represented by the histogram h0, thecharacteristic represented by the dotted line in FIG. 5B is obtained.When the luminance distribution in FIG. 5A is represented by thehistogram h1, the characteristic represented by the solid line in FIG.5B is obtained. In this case, β represents a value with reference to thelinear characteristic represented by the dotted line (β=1). Changing βin accordance with an input level can obtain a characteristic like thatrepresented by the solid line in FIG. 5B. In the case shown in FIG. 5B,β is 1 at LV2, whereas β is a value smaller than 1 at LV1 and a valuelarger than 1 at LV3. As described above, in the case of the histogramh1 in FIG. 5A, the input/output level characteristic is controlled bythe gain control value β such that the input/output characteristic curvein the range from LV1 to LV3, in which the luminance distribution islarge, has a steeper slope than the slopes in other regions. Obtainingthe corrected image signal 13 with such a characteristic will assignmore output luminance levels to regions in an image in whichdistributions are large, thereby obtaining an image with goodvisibility.

FIGS. 5C to 5F are graphs for explaining an example of control when theluminance distributions are different from the distribution shown inFIG. 5A.

First of all, FIG. 5C shows an example of a histogram when luminancedistribution at luminance levels equal to or lower than LV2 is largerthan that at luminance levels equal to or higher than LV2. FIG. 5D showsan example of the gain control value β in this case. As shown in FIG.5D, more output luminance levels are assigned to a luminance band with alarge image distribution by controlling the gain control value β suchthat the slope of the characteristic curve at luminance levels equal toor lower than LV2, at which the luminance distribution is large, is madesteeper than that at luminance levels equal to or higher than LV2. Thismakes it possible to obtain an image with good visibility.

Next, FIG. 5E shows an example of a histogram when luminancedistribution at luminance levels equal to or higher than LV2 is largerthan that at luminance levels equal to or lower than LV2. FIG. 5F showsan example of the gain control value β in this case. As shown in FIG.5F, more output luminance levels are assigned to a luminance band with alarge image distribution by controlling the gain control value β suchthat the slope of the characteristic curve at luminance levels equal toor higher than LV2, at which the luminance distribution is large, ismade steeper than that at luminance levels equal to or lower than LV2.This makes it possible to obtain an image with good visibility.

With the series of control operations by the image combining unit 26described above, it is possible to obtain a corrected image havingadvantages of both Retinex processing using a McCann99 model excellingin illumination light estimation performance and Retinex processingusing an MSR model excelling in contrast correction performance, whileobtaining good visibility.

Note that the above combination of the Retinex models in the abovedescription is not exhaustive, and a combination of Retinex models basedon different methods may be used. In addition, models to be combined arenot limited to models based on two methods, and three or more models maybe combined. In this case, the plurality of Retinex processing unitsshown in FIG. 2 may be arranged in parallel, and the image combiningunit 26 may combine corrected images from the respective Retinexprocessing units to obtain the corrected image signal 13.

Example 2

Example 2 differs from Example 1 in the operation of the imagecorrection unit 100 in the image display device in FIG. 1. Differencesfrom Example 1 will be described below. Portions which are notspecifically described are the same as those in Example 1, and hence adescription of them will be omitted.

An image correction unit 100 according to Example 2 will be describedwith reference to FIG. 2. A first Retinex processing unit 20 and asecond Retinex processing unit 22 respectively perform image processingbased on Retinex theories based on different methods with respect to aninternal image signal 12 to output a corrected image signal 21 and acorrected image signal 23. Assume that the second Retinex processingunit 22 performs Retinex processing with a larger scale than that ofRetinex processing performed by the first Retinex processing unit 20. Inthis case, the scale of Retinex processing indicates the size of a pixelrange to be referred to in Retinex processing.

A feature analyzing unit 24 analyzes the features of the internal imagesignal 12 and outputs a first image combining control signal 29 and asecond image combining control signal 25 to an image combining unit 26.The image combining unit 26 outputs a corrected image signal 13 bycombining the corrected image signal 21 and the corrected image signal23 based on the image combining control signal 29 and the imagecombining control signal 25.

In this case, the second image combining control signal 25 and a gaincontrol value β in Example 2 are then same as those in Example 1, andhence a description of them will be omitted.

A gain control value α based on the first image combining control signal29 in Example 2 is different from that in Example 1. This value will bedescribed below.

FIG. 6 shows an example of the output characteristic of a first imagecombining control signal in the feature analyzing unit 24 in Example 2.Referring to FIG. 6, the abscissa represents the luminance level of animage, and the ordinate represents the value of the first imagecombining control signal 29. As shown in FIG. 6, for example, when theluminance level is low, α is decreased, and the opposite is true.Controlling α in this manner can change a combining ratio in accordancewith a luminance level. If the luminance level represented by thecorrected image signal 13 obtained by the image combining unit 26 islow, the ratio of the second Retinex processing unit 22 can beincreased. In addition, if the luminance level is high, the ratio of thefirst Retinex processing unit 20 can be increased. That is, since asignal from the first Retinex processing unit 20 with a small Retinexprocessing scale contains many reflected light components of relativelyhigh frequency components, the fineness of an image can be improved byincreasing the combining ratio in a high-luminance image region. Inaddition, since a signal from the second Retinex processing unit 22 witha large Retinex processing scale contains many reflected lightcomponents of relatively low frequency components, the visibility of ashadow portion of an image can be improved by increasing a combiningratio in a low-luminance image region. Note that the characteristicshown in FIG. 6 is an example, and the maximum value, the minimum value,the slope of a characteristic, and the like at each luminance level maybe determined in accordance with the characteristics of Retinexprocessing.

Example 2 described above has exemplified the case in which the imagecombining control signal 29 is generated in accordance with theluminance level of an image. However, control may be performed inaccordance with frequency components. Assume that control is performedin accordance with frequency components. In this case, if, for example,frequency components in each region of an image signal are high, theratio of an image signal, of the corrected image signal 13, which isobtained from the Retinex processing unit with a small scale size isincreased. If frequency components in each region of an image signal arelow, the ratio of an image signal, of the corrected image signal 13,which is obtained from the Retinex processing unit with a large scalesize is increased. In addition, combining control may be performed byusing both the luminance levels and frequency components of an image. Inthis case, for example, control may be performed by using a valuenormalized by adding or integrating the above control valuecorresponding to a luminance level and a control value corresponding toa frequency component.

According to Example 2 of the present invention described above, it ispossible to satisfy both the requirements for the fineness of an imageand the visibility of a shadow portion by combining corrected imagesobtained by different types of Retinex processing in accordance with thescales of the respective types of Retinex processing.

Example 3

Example 3 using different Retinex models for an image correction unit100 in the image display device shown in FIG. 1 will be described next.Although the arrangement shown in FIG. 2 is used as an example of thearrangement of the image correction unit 100, this is not exhaustive.FIG. 7 shows an example of the arrangement of a first Retinex processingunit 20, which includes a reflected light detection unit 150 whichreceives an internal image signal 12 as an input signal, and performsimage processing based on a Retinex theory to detect two reflected lightcomponents 101 and 102 and a reflected light control unit 180 whichreceives the two detected reflected light components and combines themupon adjustment of the reflected light to output a corrected imagesignal 13.

The reflected light detection unit 150 and the reflected light controlunit 180 will be described next.

Light reflections are classified as follows in accordance with theproperties of objects, for example: light (to be referred to as specularhereinafter) specularly reflected by a smooth surface like a mirror;light (to be referred to as diffuse hereinafter) diffusely reflected bymicroscopic asperities on a rough surface; and ambient light (to bereferred to as ambient hereinafter) scattered upon repeated reflectionand the like by a surrounding environment.

In the field of three-dimensional computer graphics, for example, thereis available a Phong reflection model as a reflection model expressingthe shadow of the surface of an object by using the properties of thesethree types of light. According to the Phong reflection model, amaterial can be expressed by the manner of light reflection.

For example, when spot light is applied to a plastic sphere, ahigh-luminance circular small highlight is formed. When spot light isapplied to a rubber sphere, a highlight has a larger radius than that onthe plastic sphere but has a lower luminance. This highlight portion isspecular. In addition, diffuse and ambient differ in luminance dependingon materials.

FIG. 10 is a view for explaining an example of a Phong reflection model.FIG. 10 is constituted by a light source, light beams extending from thelight source, a sphere at which the light beams have arrived, a floor onwhich the sphere is placed, and an observer who observes this condition.Observation is performed at the position of a viewpoint, and may beperformed by the naked eye or through an observation device such as acamera.

Referring to FIG. 10, the specular is light 501 which is a light beamreflected by the sphere surface along the line of sight, and is areflection of the light source in the sphere surface. A circularhighlight 504 in FIG. 10 indicates the range of the specular. In thecase of a plastic sphere, for example, a high-luminance circular smallhighlight is formed. In the case of a rubber sphere, a highlight isformed to have a larger radius and lower luminance than that formed on aplastic sphere. The Phong reflection model assumes that specular willcomply with the power of the cosine between the line of sight andreflected light.

Referring to FIG. 10, diffuse is light 502 which is a light beamincident on the sphere surface and diffused and reflected. The luminanceof diffuse is determined by the directions of a light beam and thesphere surface, that is, the cosine between the light beam and thenormal line, and hence a portion, of the sphere surface, which lightdirectly strikes is the range of the diffuse.

Referring to FIG. 10, ambient is light 503 which goes around a shadowportion. This light appears when light repeatedly reflected andscattered by the surrounding is averaged throughout the environment andstays there, and hence even a shadow portion which light does notdirectly reach has a constant luminance. Ambient is diffused/reflectedlight forming a shadow. The brightness of ambient is determined by thedirections of a light beam and the sphere surface, that is, the cosinebetween a light beam vector and the normal line.

As described above, the Phong reflection model is expressed by thefollowing equation.

$\begin{matrix}{I = {{k_{d}{\sum\limits_{j = 1}^{l}{\left( {\overset{\rightarrow}{N} \cdot \overset{\rightarrow}{L}} \right)m_{d}}}} + {k_{s}{\sum\limits_{j = 1}^{l}{\left( {\overset{\rightarrow}{R} \cdot \overset{\rightarrow}{V}} \right)^{n}I_{j}}}} + {I_{a}.}}} & \left\lbrack {{Math}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Assume therefore that reflected light in the reflected light detectionunit in Example 3 is constituted by ambient, diffuse, and specular, andthat ambient in an image complies with a wide-scale Gaussiandistribution, diffuse complies with a luminance distribution in the formof a cosine distribution, and specular complies with a luminancedistribution in the form of a raised cosine distribution. Letting Fa(x,y) represent an ambient filter, Fd (x, y) represents a diffuse filter,and Fs(x, y) represents a specular filter, the respective filters areexpressed by the following equations.

$\begin{matrix}{{F_{a}\left( {x,y} \right)} = {\frac{1}{\sqrt{2\;\pi}\sigma}e^{- \frac{x^{2} + y^{2}}{2\sigma^{2}}}}} & \left\lbrack {{Math}\mspace{14mu} 2} \right\rbrack \\{{F_{d}\left( {x,y} \right)} = {{\cos\left( \frac{\pi\sqrt{x^{2} + y^{2}}}{k} \right)}/N}} & \left\lbrack {{Math}\mspace{14mu} 3} \right\rbrack \\{{F_{s}\left( {x,y} \right)} = {{\cos^{n}\left( \frac{\pi\sqrt{x^{2} + y^{2}}}{k} \right)}/N^{n}}} & \left\lbrack {{Math}\mspace{14mu} 4} \right\rbrack\end{matrix}$

In addition, FIGS. 11A, 11B, and 11C are graphs for respectivelyexplaining ambient, diffuse, and specular distributions, with eachordinate representing the luminance level, and each abscissarepresenting the one-dimensional positional coordinates. As is obvious,the diffuse and specular distributions decrease in level steeply ascompared with the Gaussian distribution of ambient.

In this case, an image Ia obtained by the ambient filter contains almostonly ambient components because of the averaging of overall light. Animage Id obtained by the diffuse filter contains almost only ambient anddiffuse components because of the averaging of specular components bythe filter. In an image Is obtained by the specular filter, all ambient,diffuse, and specular components are left because they are hardlyaveraged. This state is expressed by equations (5).Ambient=I _(a),Diffuse=I _(d) −I _(a),Specular=I _(s) −I _(d)  [Math 5]

Using this, logarithmic-space reflection components are calculated inthe same manner as in MSR according to equations 6.R _(Phong,i)(x,y)=W _(d) R _(Diffuse,i)(x,y)+W _(s) R _(Specular,i)(x,y)R _(Specular,i)(x,y)=log Ī _(s,i)(x,y)−log Ī _(d,i)(x,y)=log [F_(s)(x,y)

I(x,y)]−log [F _(d)(x,y)

I(x,y)]R _(Diffuse,i)(x,y)=log Ī _(d,t)(x,y)−log Ī _(a,i)(x,y)=log [F _(d)(x,y)

I(x,y)]−log [F _(a)(x,y)

I(x,y)]  [Math 6]

Specular from a mirror, metal, or the like is considered to be totalreflection, and hence a raised cosine value becomes infinite. At thistime, the specular reflection component may also be represented byEquation 7.R _(specular,i)(x,y)=log I(x,y)−log [F _(d)(x,y)

I(x,y)]=log I _(i)(x,y)−log Ī _(d,i)(x,y)  [Math 7]

Since ambient is average light in the entire environment, either anaverage value filter or an average luminance may be used in place of aGaussian filter. For example, when using the average luminance, Equation8 is given.R _(Diffuse,i)(x,y)=log Ī _(d,i)(x,y)−log [ΣI(x,y)/num]=log Ī_(d,i)(x,y)−log Ī _(a,i)(x,y)  [Math 8]

Specular is noticeable because it is often a high-luminance highlight,and diffuse often has middle or low luminance. For this reason, forexample, the gain of a high-luminance region like that shown in FIG. 12Amay be added to the specular Rspecular of Equations 6, whereas the gainof a middle/low-luminance region like that shown in FIG. 12B may beadded to the diffuse Rdiffuse. In this case, letting g(l) be aninput/output curve in FIG. 12A, the gain becomes 0 when the inputluminance 1 is low. The gain gradually increases from middle luminance.At high luminance, the gain becomes 1. The input/output curve in FIG.12B is represented by 1-g(l). When the luminance is low, the gain is 1.The gain gradually decreases from middle luminance. At high luminance,the gain becomes 0.

In a similar manner to the case of MSR, Equations 6 represent ahomomorphic filter when adding a gain and exponential function afterexecution of the weighted averaging. For this homomorphic filter, thelogarithmic and exponential functions may be approximated by, forexample, a power-used function and its inverse function. In this case,when using a function ƒ, Equations 9 are given.R _(Phong,i)(x,y)=W _(d) R _(Diffuse,i)(x,y)+W _(s) R _(Specular,i)(x,y)R _(Specular,i)(x,y)=ƒ(F _(s)(x,y)

I(x,y))−ƒ(F _(d)(x,y)

I(x,y))−ƒ(Ī _(s,i)(x,y))−ƒ(Ī _(d,i)(x,y))R _(Diffuse,i)(x,y)=ƒ(F _(d)(x,y)

I(x,y))−

(F _(a)(x,y)

I(x,y))−ƒ(Ī _(d,i)(x,y))−ƒ(Ī _(a,i)(x,y))  [Math 9]

As described above, it is possible to perform correction by using thePhong reflection model in consideration of the properties of reflection.

Equations (9) will be described with reference to FIGS. 8 and 9.

FIG. 8 is a block diagram for explaining processing by the reflectedlight detection unit according to Example 3. The reflected lightdetection unit 150 includes a specular filter unit 151, a diffuse filterunit 153, an ambient filter unit 155, function transformation units 157,159 and 161, a specular detection unit 163, and a diffuse detection unit164. Note that the function transformation units may be logarithmfunctions or may be approximated by power functions.

FIG. 9A is a block diagram for explaining processing by the reflectedlight control unit according to Example 1. The reflected light controlunit 180 may include weighted averaging units using weights W1 and W2 ormay include the weighted averaging units using weights W1 and W2, a gainG, and an inverse function transformation unit 182. Note however thatthe inverse function transformation unit is the inverse function of thefunction used in the function transformation unit. In addition, as shownin FIG. 9B, the arrangement shown in FIG. 9A may additionally include aspecular correction gain 183 having a high gain in the high luminanceregion in FIG. 12A and a diffuse correction gain 184 having a high gainin the middle/low luminance region in FIG. 12B.

According to the arrangement described above, when extracting reflectedlight components, it is possible to obtain the first corrected imagesignal 21 representing high texture quality in consideration of thematerial of an object in an image from the first Retinex processing unit20 by decomposing the image for the respective properties of lightreflection, i.e., for specular, diffuse, and ambient, and changing thecorrection amount in accordance with the respective properties.

Assume then that the second Retinex processing unit 22 performs imagecorrection using an MSR model. In this case, the second Retinexprocessing unit 22 performs processing using a larger scale size thanthat used by the first Retinex processing unit 20.

With the above arrangement, the first corrected image signal 21 becomesan image signal obtained in consideration of the properties of theobject, and the second corrected image signal 23 becomes an image signalobtained by contrast correction on a relatively large area of the image.These corrected image signals are combined by an operation similar tothat of the image combining unit 26 described in Example 2. With thisoperation, in a region, of the image, in which the luminance level ofthe image is low, the ratio of the second corrected image signal can beincreased, and hence the contrast improvement effect can be increased.In a region in which the luminance level of the image is high, since theratio of an image corrected signal obtained in consideration of theproperties of the object can be increased, and hence it is possible toobtain an image with good visibility throughout the entire band of theluminance levels of the image as the corrected image signal 13.

Example 3 of the present invention described above can obtain an outputimage with higher texture quality in addition to the effects of Example2 described above.

Example 4

Example 4 will exemplify adaptive control giving consideration tooutside light in an image display device according to the presentinvention in a usage environment.

FIG. 13 shows an example of the arrangement of an image display deviceaccording to Example 4.

This image display device includes an input signal processing unit 11which receives an image input signal 10 and converts it into an internalimage signal 12 by using, for example, a compressed image signaldecoder, IP conversion, or scaler, an illuminance sensor 31 whichreceives outside light and outputs, for example, an illuminance levelsignal 32 with 256 level steps, an image correction unit 300 whichreceives the internal image signal 12 and the illuminance level signal32, a timing control unit 14 which receives a corrected image signal 33and generates a display control signal 15 based on the corrected imagesignal and horizontal/vertical synchronization signals for a displayscreen, and an optical system device 200 which displays an image.

FIG. 14 shows an example of the arrangement of the image correction unit300. An image combining unit 261 adaptively combines a first correctedimage signal 21 and a second corrected image signal 23 in accordancewith a first image combining control signal 29, a second image combiningcontrol signal 25, and the illuminance level signal 32, and outputs theresultant signal as a corrected image signal 13.

FIG. 15 shows an example of the arrangement of the image combining unit261. An illuminance gain adjusting unit 262 adjusts the first imagecombining control signal 29 and the second image combining controlsignal 25 in accordance with the illuminance level signal 32. Theilluminance gain adjusting unit 262 then outputs the resultant signalsas a second illuminance-corrected signal 251 and a firstilluminance-corrected signal 291, respectively, which are then added togain control units 27, 28, and 31, as shown in FIG. 15. This determinesthe combining characteristics of the corrected image signal 13.

In the above arrangement, first of all, if, for example, the illuminanceis high, the first image combining control signal 29 is corrected inaccordance with illuminance level signal 32 so as to increase thecombining ratio of the Retinex processing unit with a small scale size,thereby output the first illuminance-corrected signal 291. That is, inthe arrangement example described in Example 2, an offset may be addedin a direction to increase the a value shown in FIG. 6.

In accordance with the magnitude of the illuminance level signal 32, thesecond image combining control signal 25 is then formed into the secondilluminance-corrected signal 251 by, for example, calculating theproduct of the signal 25 and the gain control signal shown in FIG. 16.With this operation, when the ambient illuminance is low, the gaincontrol value β decreases to reduce the correction amount of an image,thereby playing back an image close to the original image. When theilluminance is high, the gain control value increases to increase thecorrection amount of an image, thereby improving the visibility of theimage in a bright environment. This correction method is not exhaustive.For example, an offset may be added to the second image combiningcontrol signal 25 in accordance with the illuminance level signal 32.That is, control may be performed to increase the correction amount in abright environment in accordance with an illuminance level.

According to Example 4 of the present invention described above, thevisibility of an image can be improved even in a bright environment bycontrolling image processing in consideration of the influence ofoutside light.

Example 5

Example 5 will exemplify a control method when a user sets correctioncharacteristics in the image display device according to the presentinvention. FIG. 17 shows an example of the arrangement of Example 5,which is provided with a user setting unit 400. The user setting unit400 is configured to receive an operation signal 401 from the user viaan operation on an operation button on a remote controller or the devicebody and output an operation instruction signal to an image correctionunit 100 in accordance with an operation signal so as to set informationindicating whether to perform correction or a correction amount in imageprocessing in the image display device. This makes it possible to makesetting to switch an image to be displayed on a display unit to a statedesired by the user.

Although FIG. 17 exemplarily shows the arrangement of the image displaydevice in FIG. 1 provided with the user setting unit 400, this is notexhaustive. An example of arrangement of the image display device havingthe illuminance sensor shown in FIG. 13 may be provided with the usersetting unit 400. That is, Example 5 may be applied to any of Examplesdescribed above.

FIG. 18 exemplifies setting items which can be set by the user settingunit according to Example 5 using a setting menu screen 1800 displayedby the image display device.

The setting menu screen 1800 is generated by a menu screen signalgeneration unit (not shown) of the image display device and output inplace of a corrected image signal 13. Alternatively, the setting menuscreen 1800 is output while being superimposed on the corrected imagesignal 13.

The item “Retinex Method Selection” 1810 in an example of the settingmenu screen 1800 will be described. With the item “Retinex MethodSelection” 1810, it is possible to select whether it is necessary to useRetinex processing by both of the first Retinex processing unit 20 andthe second Retinex processing unit 22 described in each Example.Selection is performed by moving a cursor 1811 through an operation onan operation button on the remote controller or the device body. Aselected item and corresponding processing will be described. Forexample, when the item “Retinex 1 only” is selected, only processing bythe first Retinex processing unit 20 is applied to processing by theimage correction unit, and processing by the second Retinex processingunit 22 is not applied to the processing by the image correction unit.More specifically, a combining control value α may be set to 1, or theoperation of the second Retinex processing unit 22 itself may be turnedoff. Next, on the contrary, when the item “Retinex 2 only” is selected,only processing by the second Retinex processing unit 22 is applied toprocessing by the image correction unit, and processing by the firstRetinex processing unit 20 is not applied to the processing by the imagecorrection unit. More specifically, the combining control value α may beset to 0, or the operation of the first Retinex processing unit 20itself may be turned off. When the selection item “Combining Retinex 1and 2” is selected, processing by the first Retinex processing unit 20and processing by the second Retinex processing unit 22 are combined,and the resultant data is output as described above in Example describedabove. When the selection item “Retinex OFF” is selected, both ofprocessing by the first Retinex processing unit 20 and processing by thesecond Retinex processing unit 22 are not applied to processing by theimage correction unit. The operations of both the units may be turnedoff, or an image input to the image correction unit may be output whilebypassing the image correction unit.

In the item “Retinex Method Selection” 1810 described above, it is notalways necessary to present the above four selection items to the user.For example, only two selection items “Combining Retinex 1 and 2” and“Retinex OFF” may be presented. Alternatively, three selection items“Combining Retinex 1 and 2”, “Retinex 1 only”, and “Retinex OFF” may bepresented. That is, at least two items of the exemplified items may bepresented.

An item “Retinex Intensity Setting” 1820 of the example of the settingmenu screen 1800 will be described next. In the item “Retinex IntensitySetting” 1820, the intensity of each Retinex processing can be set. Morespecifically, the intensity of each Retinex processing is set by movingslide bars 1821 and 1822 through an operation on an operation button onthe remote controller or the device body. Processing in this case can beimplemented by, for example, adding an offset to the gain of eachRetinex processing shown in FIGS. 4A and 4B in accordance with theintensity. For example, a positive offset is added to the gain shown inFIGS. 4A and 4B when the intensity is high, and a negative offset isadded when the intensity is low. Such processing of adding the offsetcan be implemented by inserting the processing of adding the offset intothe first Retinex processing unit 20 and the second Retinex processingunit 22, or inserting it to a corrected image signal 21 and a correctedimage signal 23.

Note that the item “Retinex Intensity Setting” 1820 may be configured toswitch between an active state and an inactive state in accordance withthe selected state of the item “Retinex Method Selection” 1810. That is,the slide bar for processing turned off in the item “Retinex MethodSelection” 1810 may be the inactive state.

The item “Combining Setting” 1830 on the example of the setting menuscreen 1800 will be described next. In the item “Combining Setting”1830, the combining ratio of each Retinex processing can be set. Thisoperation is implemented by controlling the value α described in eachExample described above. More specifically, first of all, the user canselect either “Variable” or “Fixed” by moving a cursor 1831 through anoperation on an operation button on the remote controller or the devicebody. When “Variable” is selected, the combining control value α can bechanged in accordance with the input image signal as described in eachExample described above. When “Fixed” is selected, the combining controlvalue α is not changed in accordance with the input image signal, butfixed to the state selected by the user. More specifically, the useradjusts a slide bar 1832 through an operation on an operation button onthe remote controller or the device body, and the setting is made in thestate fixed to the combining control value α corresponding to theposition. In the example shown in FIG. 18, the value α is increased asthe bar moves to the left, and the combining is performed while priorityis given to processing by the first Retinex processing unit 20. Thevalue α is decreased as the bar moves to the right, and the combining ismade while priority is given to processing by the second Retinexprocessing unit 22.

Note that the item “Combining Setting” 1830 may be configured to switchbetween the active state and the inactive state in accordance with theselected state of the item “Retinex Method Selection” 1810. That is,when the item “Combining Retinex 1 and 2” is not selected, the item“Combining Setting” 1830 may be entirely set to the inactive state.

The item “Visibility Improvement Processing Intensity Setting” 1840 onthe example of the setting menu screen 1800 will be described next. Byusing the item, the magnitude of the effect of processing by the gaincontrol unit 31 in FIG. 3 can be set. More specifically, the magnitudeof the amplitude of the change amount of a gain control value β ischanged in accordance with the movement of a slide bar 1840. In all theproperties shown in FIGS. 5B, 5D, and 5F, the visibility improvementprocessing is enhanced more as the amplitude of the change amount of thegain control value β increases.

The item “Illuminance Sensor Adaptive Processing” 1850 on the example ofthe setting menu screen 1800 will be described next. The item is a menuitem used when the user setting unit 400 is provided to the arrangementexample of the image display device having the illuminance sensor shownin FIG. 13 in Example 4. In the item “Illuminance Sensor AdaptiveProcessing” 1850, the user can select either “ON” or “OFF” by moving acursor 1851 through an operation on an operation button on the remotecontroller or the device body. When “OFF” is selected, the gain controlvalue in FIG. 16 described in Example 4 is fixed to 1.

According to the image display device provided with the user settingunit 400 described in Example 5 of the present invention describedabove, the user can adjust image correction processing in each Exampleof the present invention in accordance with a user's preference, theusage purpose or usage environment of the image display device. Thismakes it possible to provide a more convenient image display device.

Example 6

Example 6 will exemplify an image display device which corrects an imageby decomposing the image for each light reflection property withreference to the arrangement of a projector. Note that the followingwill exemplify a front projector. However, a rear projection televisionmay be another form of the projector. In addition, a display deviceusing a direct-view flat-panel display designed not to perform enlargedprojection on the panel, such as a liquid crystal display, plasmadisplay, or organic EL display may be applied. This point applies to anyof Examples to be described below.

FIG. 19 shows an example of the arrangement of an image display deviceaccording to Example 6.

This image display device includes an input signal processing unit 11which receives an image input signal 10 and converts the image inputsignal into an internal image signal 601 by, for example, a compressedimage signal decoder, IP conversion, or scalar, an image correction unit1000 which receives the internal image signal 601, a timing control unit14 which receives an output image signal 606 and generates a displaycontrol signal 15 based on the corrected image signal andhorizontal/vertical synchronization signals for a display screen, and anoptical system device 200 which displays an image. Note that the imagecorrection unit 1000 may be simply expressed as an image processingunit.

The optical system device 200 includes an light source 203 which emits alight beam for projecting an image on the screen, a panel 202 whichreceives the display control signal 15, adjusts the tone of the lightbeam from the light source 203 for each pixel, and generates aprojection image, and a lens 201 for the enlarged projection of theprojection image onto the screen.

When the image display device is a direct-view flat-panel display suchas a liquid crystal display, a plasma display, or an organic EL display,the lens 201 of the optical system device 200 is not required. The userdirectly views the panel 202.

FIG. 20 shows an example of the arrangement of the image correction unit1000. A color conversion unit 602 converts the internal image signal 601input in the RGB format into, for example, a signal in the YUV format,and outputs it as an internal image signal 625. In this case, the formatof the internal image signal 625 is not limited to the YUV format aslong as it can be converted into a luminance signal Y and color signals.A first Retinex processing unit 20 and a second Retinex processing unit22 perform image processing based on the Retinex theory with respect tothe luminance signal Y of the internal image signal 625 and respectivelyoutput a first corrected image signal 21 and a second corrected imagesignal 23. Since the operations the first Retinex processing unit 20 andthe second Retinex processing unit 22 are the same as those in Example1, a detailed description of them will be omitted. A feature analyzingunit 24 analyzes the features of the internal image signal 625 andoutputs a first image combining control signal 29 and a second imagecombining control signal 25 to an image combining unit 26. The imagecombining unit 26 outputs the corrected image signal 13 by combining thecorrected image signal 21 and the corrected image signal 23 based on thefirst image combining control signal 29 and the second image combiningcontrol signal 25. The operation of the image combining unit 26 is thesame as the example described in Example 1, and a detailed descriptionof it will be omitted. A color conversion unit 603 converts a correctedimage signal 13 into a corrected image signal 604 in the RGB format byusing the luminance signal Y of the corrected image signal 13 and acolor signal UV of the internal image signal 625. At this time, a timingdifference in terms of images may occur between the corrected imagesignal 13 and the internal image signal 625 depending on the processingarrangements of the respective Retinex processing units and the imagecombining unit 26. In this case, the corrected image signal 13 isconverted into the corrected image signal 604 by correcting this timingdifference.

An output image generation unit 605 corrects the internal image signal601 based on the ratio in image signal level between the internal imagesignal 601 and the corrected image signal 604, and outputs the resultantsignal as the output image signal 606. FIG. 21 shows an example of thearrangement of the output image generation unit 605. An absolute valueobtaining unit 610 calculates the absolute value of an image level fromthe R, G, and B components of the corrected image signal 604, andoutputs an image level signal 620. An absolute value obtaining unit 611calculates the absolute values of image levels from the R, G, and Bcomponents of the internal image signal 601, and outputs an image levelsignal 621. A computing unit 612 obtains the ratio in magnitude betweenthe image level signal 620 and the image level signal 621, and outputs acorrected ratio signal 622. An image correction unit 613 outputs theoutput image signal 606 by correcting the internal image signal 601based on the corrected ratio signal 622. More specifically, the imagecorrection unit 613 corrects the internal image signal 601 bymultiplying each of the R, G, and B components by the corrected ratiosignal 622.

FIG. 22 shows an example of specific correction. FIG. 22 shows the R, G,and B levels of image signals on the R, G, and B axes in an RGB colorspace. When the R, G, and B levels of the internal image signal 601 aregiven by (r1, g1, b1), a vector V1 indicated by an arrow (solid line) isobtained. In this case, the vector V1 has the information of the ratiosamong a plurality of color parameters of the internal image signal 601.Likewise, when the levels of the corrected image signal 604 are given by(r2, g2, b2), a vector V2 indicated by an arrow (solid line) isobtained. Assume that the absolute value obtaining units 610 and 611respectively obtain the image level signals 620 and 621 by obtaining theabsolute values of the vectors from the respective R, G, and Bcomponents, i.e., the lengths of the vectors V1 and V2. The computingunit 612 obtains the ratio in magnitude between the absolute values ofthe image levels. The image correction unit 613 then performsmultiplication by using the obtained ratio.corrected image signal 606=V1*(image level 620÷image level 621)V1=(r1,g1,b1)  [Math 10]

A vector V3 indicated by the broken line in FIG. 22 represents theoutput image signal 606 obtained in this manner. That is, this vectorhas the same direction as the vector V1, and hence the ratios among theplurality of color parameters of the internal image signal 601 aremaintained. This makes it possible to obtain an image with goodvisibility while holding the same color shade as that represented by theinternal image signal 601.

This processing has been described as “corrects the internal imagesignal 601”. The processing can also be expressed as generating a newoutput image signal based on the vector of the R, G, and B levels of theinternal image signal 601, its absolute value, the vector of the R, G,and B levels of the corrected image signal 604, and its absolute value.

The image correction processing by the output image generation unit 605has been described above by taking, as an example, the correction of theinternal image signal 601 by using the ratios among the absolute valuesof the R, G, and B levels of the internal image signal 601 and thecorrected image signal 604. However, it is also possible to correct theinternal image signal 601 by using the ratio of a luminance signal (Y).

FIG. 26 shows an example of the arrangement of the image correction unit1000 in this case. The color conversion unit 602 outputs an internalimage signal 626 from the internal image signal 601 in the RGB format.Since the operations of the first Retinex processing unit 20, the secondRetinex processing unit 22, the feature analyzing unit 24, and the imagecombining unit 26 are the same as those described above, a descriptionof them will be omitted. A computing unit 631 of an output imagegeneration unit 630 obtains the ratio between the level of the correctedluminance signal 13 and the luminance signal level of the internal imagesignal 626, and outputs a corrected ratio signal 632. An imagecorrection unit 633 corrects the internal image signal 601 based on thecorrected ratio signal 632 and outputs an output image signal 608. Morespecifically, the image correction unit 633 corrects the internal imagesignal 601 by multiplying each of the R, G, and B components by thecorrected ratio signal 632. This arrangement example can also obtain animage with good visibility while holding the same color shade as thatrepresented by the internal image signal 601.

The image display device according to Example 6 described above cangenerate a new image signal based on information concerning the color ofan image signal before image correction by Retinex processing (vectorinformation in the color space or the information of the ratios among aplurality of color parameters) and the information of ratios amongabsolute values or luminances of color space vectors before and afterimage correction by Retinex processing. This makes it possible togenerate and display an image signal close to the color balance beforeRetinex processing while obtaining the effect of improving visibility bymeans of Retinex processing.

Example 7

Example 7 will exemplify a case in which the image display deviceaccording to Example 6 of the present invention is configured toadditionally perform adaptive control in consideration of outside lightin a usage environment.

FIG. 23 shows an example of the arrangement of an image display deviceaccording to Example 7.

This image display device includes an input signal processing unit 11which receives an image input signal 10 and converts it into an internalimage signal 601 by using, for example, a compressed image signaldecoder, IP conversion, or scaler, an illuminance sensor 31 whichreceives outside light and outputs, for example, an illuminance levelsignal 32 with 256 level steps, an image correction unit 3000 whichreceives the internal image signal 601 and the illuminance level signal32, a timing control unit 14 which receives an output image signal 608and generates a display control signal 15 based on the corrected imagesignal and horizontal/vertical synchronization signals for a displayscreen, and an optical system device 200 which displays an image. Notethat the image correction unit 3000 may be expressed as an imageprocessing unit.

FIG. 24 shows an example of the arrangement of the image correction unit3000. An output image generation unit 607 adaptively combines acorrected image signal 604, the internal image signal 601, and theilluminance level signal 32, and outputs the resultant signal as theoutput image signal 608. Note that the same reference numerals as thosein FIG. 20 described in Example 6 denote constituent elements having thesame functions. FIG. 25 shows an example of the arrangement of theoutput image generation unit 607. An image correction unit 615 performscorrection by using the internal image signal 601, a corrected ratiosignal 622, and the illuminance level signal 32 in the following manner,and outputs the output image signal 608.corrected ratio signal 622=image level 620÷image level 621output image signal 608=V1*(corrected ratio signal 622*(maximumilluminance÷illuminance level signal 32))V1=(r1,g1,b1)  [Math 11]

Assume that in the above description, the illuminance level increases invalue in proportion to brightness, the maximum illuminance expressed byequations 11 is the maximum illuminance level set in advance, and theilluminance level signal 32 does not exceed the maximum illuminancelevel. According to processing by the image correction unit 615, whenthe illuminance level is high, a large correction amount can be set forthe internal image signal 601, whereas when the illuminance level islow, a small correction amount can be set for the internal image signal601. This makes it possible to improve visibility by changing the imagecorrection amount in accordance with the brightness of a place whereimage display is performed. For example, increasing the image correctionamount in a bright place can obtain the effect of further improvingvisibility.

This processing has been described as “corrects the internal imagesignal 601”. The processing can also be expressed as generating a newoutput image signal based on the vector of the R, G, and B levels of theinternal image signal 601, its absolute value, the vector of the R, G,and B levels of the corrected image signal 604, and its absolute value.

The image display device according to Example 7 described above cangenerate a new image signal based on information concerning the color ofan image signal before image correction by Retinex processing (vectorinformation in the color space or the information of the ratios among aplurality of color parameters) and the information of ratios amongabsolute values or luminances of color space vectors before and afterimage correction by Retinex processing. This can further adjust theeffect of improving visibility in accordance with the brightness of aplace where image display is performed. This makes it possible togenerate and display an image signal close to the color balance beforeRetinex processing while more suitably obtaining the effect of improvingvisibility by means of Retinex processing.

Example 8

An image display device according to Example 8 is obtained by making theimage correction unit 1000 of the image display device according toExample 6 of the present invention shown in FIG. 19 have an internalarrangement like that shown in FIG. 27.

A first Retinex processing unit 20 in FIG. 27 performs the sameprocessing as that performed by the first Retinex processing unit 20described in Example 3. As described in Example 3, Retinex processing isperformed by using the Phong reflection model expressing the shadow ofan object surface by using the properties of three kinds of light. Asshown in FIG. 27, the image display device according to Example 8performs image signal generation processing performed by an output imagegeneration unit 605. A color conversion unit 602 converts an internalimage signal 601 input in the RGB format into, for example, a signal inthe YUV format, and outputs it as an internal image signal 625. In thiscase, the format of the internal image signal 625 is not limited to theYUV format as long as it can be converted into a luminance signal Y andcolor signals. The first Retinex processing unit 20 performs imageprocessing based on the Retinex theory with respect to the luminancesignal Y of the internal image signal 625, and outputs a corrected imagesignal 21. A color conversion unit 603 converts the corrected imagesignal 21 into a corrected image signal 604 in the RGB format by usingthe luminance signal Y of the corrected image signal 21 and a colorsignal UV of the internal image signal 625. At this time, a timingdifference in terms of images may occur between the corrected imagesignal 21 and the internal image signal 625 depending on the processingarrangements of the respective Retinex processing units and an imagecombining unit 26. In this case, the corrected image signal 21 isconverted into the corrected image signal 604 by correcting this timingdifference.

The output image generation unit 605 corrects the internal image signal601 based on the ratio in image signal level between the internal imagesignal 601 and the corrected image signal 604, and outputs the resultantsignal as an output image signal 606. This processing is the same asthat in Example 6, and the ratio in image level is obtained by using theabsolute value of the color space vector of the R, G, and B levels ofeach image signal or the Y signal of each image signal. According toExample 8, it is possible to obtain the output image signal 606 withhigh texture quality in consideration of the material of an object in aninput image while holding the same color shade of the object.

This processing has also been described as “corrects the internal imagesignal 601” in Example 8. The processing can also be expressed asgenerating a new output image signal based on the vector of the R, G,and B levels of the internal image signal 601, its absolute value, thevector of the R, G, and B levels of the corrected image signal 604, andits absolute value.

The image display device according to Example 8 described above can alsogenerate a new image signal based on information concerning the color ofan image signal before image correction by Retinex processing (vectorinformation in the color space or the information of the ratios among aplurality of color parameters) and the information of ratios amongabsolute values or luminances of color space vectors before and afterimage correction by Retinex processing. This makes it possible togenerate and display an image signal closer to the color balance beforeRetinex processing while more suitably obtaining the effect of improvingvisibility by means of Retinex processing. In addition, the imagedisplay device according to Example 8 can be implemented by anarrangement simpler than that of Example 6.

Note that the arrangement according to Example 8 may be provided with anilluminance sensor 31 as in Example 7 to additionally perform controlusing an illuminance level signal 32.

In addition, each of the arrangements according to Examples 6, 7, and 8may be provided with a user setting unit 400 like the one in Example 5to display a setting menu screen 1800 in FIG. 18 in accordance with anoperation signal 401 from the user and allow user settings in therespective items.

REFERENCE SIGNS LIST

-   -   10 . . . image input signal    -   12 . . . internal image signal    -   13 . . . corrected image signal    -   15 . . . display control signal    -   20 . . . first Retinex processing unit    -   21 . . . first corrected image signal    -   22 . . . second Retinex processing unit    -   23 . . . second corrected image signal    -   24 . . . feature analyzing unit    -   25 . . . image combining control signal    -   26 . . . image combining unit    -   27, 28, 31 . . . gain control unit    -   29 . . . image combining control signal    -   30 . . . addition unit    -   32 . . . illuminance level signal    -   33 . . . corrected image signal obtained by adaptive control    -   100 . . . image correction unit    -   101 . . . reflected light component based on scale 1    -   102 . . . reflected light component based on scale 2    -   120 . . . reflected light detection unit based on MSR    -   122 . . . result of convolution product obtained by scale 1        filter    -   124 . . . result of convolution product obtained by scale 2        filter    -   126 . . . resultant value obtained by SSR with scale 1    -   128 . . . resultant value obtained by SSR with scale 2    -   130 . . . reflected light control unit based on MSR    -   131 . . . weighted average resultant value (including gain)        based on respective SSR results    -   152 . . . result of convolution product obtained by specular        filter    -   154 . . . result of convolution product obtained by diffuse        filter    -   156 . . . result of convolution product obtained by ambient        filter    -   158 . . . result of function transformation with respect        specular filter    -   160 . . . result of function transformation with respect diffuse        filter    -   162 . . . result of function transformation with respect ambient        filter    -   181 . . . weighted average resultant value (including gain) with        respect to specular and diffuse    -   302 . . . edge signal    -   601 . . . internal image signal    -   602, 603 . . . color conversion unit    -   604 . . . corrected image signal    -   605, 607, 630 . . . output image generation    -   606, 608 . . . output image signal    -   610, 611 . . . absolute value obtaining unit    -   612, 631 . . . computing unit    -   613, 615, 633 . . . image correction unit    -   1000, 3000 . . . image correction unit

The invention claimed is:
 1. An image display device characterized bycomprising: an image input unit which inputs an image signal; an imageprocessing unit which performs Retinex processing with respect to theinput image signal and performs image signal generation to generate anew image signal based on information concerning a color of the inputimage signal and information of an absolute value or Y-value of a colorspace vector of the input image signal having undergone the Retinexprocessing; and a display unit configured to display a first image basedon the new image signal generated by said image processing unit, whereinthe image processing unit performs, as the Retinex processing, firstRetinex processing for the input image signal and second Retinexprocessing, based on a method different from the first Retinexprocessing, for the input image signal, and combining a first imagesignal output from the first Retinex processing and a second imagesignal output from the second Retinex processing in accordance withfeatures of the input image signal to generate the new image signal,wherein the display unit is further configured to display a setting menuscreen, wherein the setting menu screen is configured to switch thedisplay unit between at least two of a plurality of display statesincluding: a first state in which the first image based on the new imagesignal is displayed, a second state in which a second image based on thefirst image signal output from the first Retinex processing withouthaving undergone the second Retinex processing is displayed, a thirdstate in which a third image based on the second image signal outputfrom the second Retinex processing without having undergone the firstRetinex processing is displayed, and a fourth state in which a fourthimage based on the input image signal which has not undergone the firstRetinex processing and the second Retinex processing is displayed. 2.The image display device according to claim 1, wherein the new imagesignal generated by said image processing unit includes a vector in acolor space which has a same direction as that of a vector in a colorspace of the input image signal or includes color parameters whoseratios are the same as color parameters of the input image signal, andthe absolute value or Y-value of the color space vector of the inputimage signal is the same as an absolute value or Y-value of a colorspace vector of the new image signal.
 3. The image display deviceaccording to claim 1, wherein the setting menu screen is furtherconfigured to set a combining ratio between the first image signal andthe second image signal, and the first image signal output from thefirst Retinex processing and the second image signal output from thesecond Retinex processing are combined in accordance with the setcombining ratio.
 4. The image display device according to claim 1,wherein a scale in the first Retinex processing is different from ascale in the second Retinex processing.
 5. The image display deviceaccording to claim 1, wherein a combining ratio between the first imagesignal output from the first Retinex processing and the second imagesignal output from the second Retinex processing is changed inaccordance with luminance information or frequency information of theinput image signal.
 6. The image display device according to claim 4,wherein the scale in the first Retinex processing is smaller than thescale in the second Retinex processing, when a luminance of the inputimage signal is higher than a predetermined level, a combining ratio ofthe first image signal having undergone the first Retinex processing isset to be larger than that of the second image signal having undergonethe second Retinex processing, and when a luminance of the input imageis lower than the predetermined level, a combining ratio of the firstimage signal having undergone the first Retinex processing is set to besmaller than that of the second image signal having undergone the secondRetinex processing.
 7. The image display device according to claim 1,further comprising: an illuminance sensor which measures an illuminanceof outside light, wherein the combining of the first image signal outputfrom the first Retinex processing and the second image signal outputfrom the second Retinex processing is changed in accordance with ameasurement result obtained by said illuminance sensor.
 8. The imagedisplay device according to claim 1, wherein the first Retinexprocessing comprises processing of separating the input image signalinto a plurality of reflected light components, adjusting each of theplurality of separated reflected light components based on a weightvalue, performing weighted averaging, and controlling a ratio ofreflected light in the first image signal, and wherein the secondRetinex processing uses a larger scale than the first Retinexprocessing.